This article introduces the principles, implementation and application of the Wide & Deep model and DCN (Deep & Cross Network) model in processing feature learning. The article describes in detail the simple linear transformation of the Wide part and the multi-layer perceptron structure of the Deep part, and explains how to improve the Wide part through Cross Network to explicitly model the intersection between features....

This article deduces the principle of XGBoost in detail, including the definition of loss function, optimization of objective function and regularization method. The article also introduces XGBoost’s training methods and common difficulties, such as slow training speed, parallelization issues, high-dimensional feature processing and over-fitting mitigation, etc....

This article provides an in-depth exploration of collaborative filtering algorithms, including two fundamental approaches—User CF and Item CF—along with their respective advantages and disadvantages. Additionally, it offers a detailed discussion on the application of matrix factorization techniques in recommendation systems, covering the basic principles, strengths, and weaknesses of eigenvalue decomposition, singular value decomposition (SVD), and gradient descent methods. Furthermore, the article highlights the current challenges faced by collaborative filtering models and potential directions for improvement....

This article introduces the FM model, which is a statistical model used to deal with high-dimensional sparse data and feature intersection, and can capture the relationship between features. The FM model reduces the number of computational parameters and increases the capture of feature interactions by introducing low-rank factor vectors, paying special attention to second-order interactions. The model uses gradient descent to optimize the loss function and learns the interaction between users and items through factor vectors. In recommendation systems, FM models can be used to predict users' ratings or click probabilities of items by analyzing the interaction of features between users and items....

This article analyzes the reasons why MSE is commonly used instead of CE for classification tasks in machine learning, and discusses it in detail from the perspective of distribution and optimization. The article points out that MSE assumes that data obeys Gaussian distribution, but in reality long-tailed distributions are also common, and using MSE directly may lead to poor results. At the same time, the article also mentioned that from an optimization perspective, if the classification problem uses MSE as the loss function, it is a non-convex optimization and it is difficult to find the optimal solution....

This is a blog post on the implementation of DecisionTree algorithm using C++....

AI Summary: This article introduces the fundamental principles of LightGBM, highlights of its engineering implementation, and improvements over XGBoost. LightGBM is a gradient boosting decision tree (GBDT) model developed by Microsoft, characterized by fast training speed and low memory usage. The article elaborates on technical details such as the histogram approximation method, GOSS (Gradient-based One-Side Sampling), and EFB (Exclusive Feature Bundling), and compares its differences from XGBoost....